The forming of virtual reality (VR) environments has become possible in the recent past. In such VR environments, a user is provided with impressions, which are perceived to be real. Concepts for rendering a real object in a VR environment are described in an international patent application WO-01/93209, publication date Dec. 6, 2001, by Gross et al., “Method for the production of a virtual reality environment.” There, concepts for the concurrently acquiring and rendering 3D image data are described.
The three-dimensional integration of real objects into VR environments entails great challenges with regards to image processing. An interaction of the user with rendered objects is only useful if it occurs in real-time.
Although real-time image acquisition and rendering have been available for 2D representation, the embedding and reproduction of real 3D objects into VR environments has been limited. Although image acquisition and image reconstruction of real objects have been accomplished, the rendering of 3D video data is still in its infancy. This is due to the fact that a series of problems intermingle. One must first acquire 3D information from primary data—generally 2D video images, and these data must be represented and transmitted in a suitable data structure, and finally a high quality rendering must be produced. Existing methods are typically based on triangle meshes or acquired images.
According to mesh technology, video data are acquired and arranged as a mesh of triangles. Rendering meshes has the advantage that existing rendering hardware, e.g., graphic cards, are mostly triangle-based. This has a positive effect on the processing speed. Image-based reproduction essentially renders directly from acquired images with interpolation between camera views. Interpolating between views increases the complexity and decreases the quality.
Therefore, there exists the need for an efficient method for acquiring, processing and rendering 3D video information.